Random Constraint Satisfaction Problems Research Articles

Free energy subadditivity for symmetric random Hamiltonians

We consider a random Hamiltonian H:Σ→R defined on a compact space Σ that admits a transitive action by a compact group G. When the law of H is G-invariant, we show its expected free energy relative to the unique G-invariant probability measure on Σ, which obeys a subadditivity property in the law of H itself. The bound is often tight for weak disorder and relates free energies at different temperatures when H is a Gaussian process. Many examples are discussed, including branching random walks, several spin glasses, random constraint satisfaction problems, and the random field Ising model. We also provide a generalization to quantum Hamiltonians with applications to the quantum Sherrington–Kirkpatrick and Sachdev–Ye–Kitaev models.

PTAS for Sparse General-valued CSPs

We study polynomial-time approximation schemes (PTASes) for constraint satisfaction problems (CSPs) such as Maximum Independent Set or Minimum Vertex Cover on sparse graph classes. Baker’s approach gives a PTAS on planar graphs, excluded-minor classes, and beyond. For Max-CSPs, and even more generally, maximisation finite-valued CSPs (where constraints are arbitrary non-negative functions), Romero, Wrochna, and Živný [SODA’21] showed that the Sherali-Adams LP relaxation gives a simple PTAS for all fractionally-treewidth-fragile classes, which is the most general “sparsity” condition for which a PTAS is known. We extend these results to general-valued CSPs, which include “crisp” (or “strict”) constraints that have to be satisfied by every feasible assignment. The only condition on the crisp constraints is that their domain contains an element that is at least as feasible as all the others (but possibly less valuable). For minimisation general-valued CSPs with crisp constraints, we present a PTAS for all Baker graph classes—a definition by Dvořák [SODA’20] that encompasses all classes where Baker’s technique is known to work, except for fractionally-treewidth-fragile classes. While this is standard for problems satisfying a certain monotonicity condition on crisp constraints, we show this can be relaxed to diagonalisability —a property of relational structures connected to logics, statistical physics, and random CSPs.

A continuous constraint satisfaction problem for the rigidity transition in confluent tissues

Models of confluent tissues are built out of tessellations of the space (both in two and three dimensions) in which the cost function is constructed in such a way that individual cells try to optimize their volume and surface in order to reach a target shape. At zero temperature, many of these models exhibit a rigidity transition that separates two phases: a liquid phase and a solid (glassy) phase. This phenomenology is now well established but the theoretical understanding is still not complete. In this work we consider an exactly soluble mean field model for the rigidity transition which is based on an abstract mapping. We replace volume and surface functions by random non-linear functions of a large number of degrees of freedom forced to be on a compact phase space. We then seek for a configuration of the degrees of freedom such that these random non-linear functions all attain the same value. This target value is a control parameter and plays the role of the target cell shape in biological tissue models. Therefore we map the microscopic models of cells to a random continuous constraint satisfaction problem with equality constraints. We argue that at zero temperature, the rigidity transition corresponds to the satisfiability transition of the problem. We also characterize both the satisfiable (SAT) and unsatisfiable (UNSAT) phase. In the SAT phase, before reaching the rigidity transition, the zero temperature SAT landscape undergoes an replica symmetry breaking (RSB)/ergodicity breaking transition of the same type as the Gardner transition in amorphous solids. By solving the RSB equations we compute the SAT/UNSAT threshold and the critical behavior around it. In the UNSAT phase we also compute the average shape index as a function of the target one and we compare the thermodynamical solution of the model with the results of the numerical greedy minimization of the corresponding cost function.

Generating hard satisfiable instances by planting into random constraint satisfaction problem model with growing constraint scope length

We propose generating hard satisfiable constraint satisfaction problem (CSP) instances by planting into the k-CSP model, which is a random CSP model with growing constraint scope length. The phase diagram of the planted k-CSP model is drawn in this paper, which has no differences from that of the pure random k-CSP model, except that the planted k-CSP model has a planted cluster while the pure random k-CSP model does not. The planted cluster, which is composed of the planted solution and solutions around it, is found to be very small. This phase diagram guarantees that the planted k-CSP instances are as hard as the pure random k-CSP instances before the satisfiability (by configurations outside the planted cluster) transition occurs, and hard satisfiable instances appear near the satisfiable transition point. Experiments on instances with 102 variables support that the model can generate hard satisfiable instances.

Noisy tensor completion via the sum-of-squares hierarchy

In the noisy tensor completion problem we observe m entries (whose location is chosen uniformly at random) from an unknown n_1 times n_2 times n_3 tensor T. We assume that T is entry-wise close to being rank r. Our goal is to fill in its missing entries using as few observations as possible. Let n = max (n_1, n_2, n_3). We show that if m > rsim n^{3/2} r then there is a polynomial time algorithm based on the sixth level of the sum-of-squares hierarchy for completing it. Our estimate agrees with almost all of T’s entries almost exactly and works even when our observations are corrupted by noise. This is also the first algorithm for tensor completion that works in the overcomplete case when r > n, and in fact it works all the way up to r = n^{3/2-epsilon }. Our proofs are short and simple and are based on establishing a new connection between noisy tensor completion (through the language of Rademacher complexity) and the task of refuting random constraint satisfaction problems. This connection seems to have gone unnoticed even in the context of matrix completion. Furthermore, we use this connection to show matching lower bounds. Our main technical result is in characterizing the Rademacher complexity of the sequence of norms that arise in the sum-of-squares relaxations to the tensor nuclear norm. These results point to an interesting new direction: Can we explore computational vs. sample complexity tradeoffs through the sum-of-squares hierarchy?

The solution space structure of planted constraint satisfaction problems with growing domains

Planting a solution into the random revised B (RB) model, which is a prototype of the random constraint satisfaction problem (CSP) with growing domains, can generate very hard satisfiable CSP benchmarks. We study the solution space structure of the planted RB model. In the thermodynamic limit, by the first moment method we find that this model goes through four phase transitions as the constraint density increases. In the replica symmetric phase, what we call the independent phase transition occurs, after which the planted cluster (cluster containing the planted solution) is separated from the giant cluster. From then on, the solution space except the planted cluster goes through the same clustering phase transition and the same satisfiability phase transition as on the random RB model. The planted cluster goes through the isolated phase transition, after which the planted cluster contains only one solution. When the instances have about 102 variables, experiments show that the last three phase transitions have already appeared as in the thermodynamic limit. This phase diagram provides strong evidence that this model can generate very hard satisfiable CSP benchmarks. Finally, we find when the constraint density goes to infinity, there is a single smooth valley in the energy landscape.

A residual-based message passing algorithm for constraint satisfaction problems

Message passing algorithms, whose iterative nature captures complicated interactions among interconnected variables in complex systems and extracts information from the fixed point of iterated messages, provide a powerful toolkit in tackling hard computational tasks in optimization, inference, and learning problems. In the context of constraint satisfaction problems (CSPs), when a control parameter (such as constraint density) is tuned, multiple threshold phenomena emerge, signaling fundamental structural transitions in their solution space. Finding solutions around these transition points is exceedingly challenging for algorithm design, where message passing algorithms suffer from a large message fluctuation far from convergence. Here we introduce a residual-based updating step into message passing algorithms, in which messages with large variation between consecutive steps are given high priority in the updating process. For the specific example of model RB (revised B), a typical prototype of random CSPs with growing domains, we show that our algorithm improves the convergence of message updating and increases the success probability in finding solutions around the satisfiability threshold with a low computational cost. Our approach to message passing algorithms should be of value for exploring their power in developing algorithms to find ground-state solutions and understand the detailed structure of solution space of hard optimization problems.

Learning variable ordering heuristics for solving Constraint Satisfaction Problems

Backtracking search algorithms are often used to solve the Constraint Satisfaction Problem (CSP), which is widely applied in various domains such as automated planning and scheduling. The efficiency of backtracking search depends greatly on the variable ordering heuristics. Currently, the most commonly used heuristics are hand-crafted based on expert knowledge. In this paper, we propose a deep reinforcement learning based approach to automatically discover new variable ordering heuristics that are better adapted for a given class of CSP instances, without the need of relying on hand-crafted features and heuristics. We show that directly optimizing the search tree size is not convenient for learning, and propose to optimize the expected cost of reaching a leaf node in the search tree. To capture the complex relations among the variables and constraints, we design a representation scheme based on Graph Neural Network that can process CSP instances with different sizes and constraint arities. Experimental results on random CSP instances show that on small and medium sized instances, the learned policies outperform classical hand-crafted heuristics with smaller search tree (up to 10.36% reduction). Moreover, without further training, our policies directly generalize to instances of larger sizes and much harder to solve than those in training, with even larger reduction in the search tree size (up to 18.74%).

The number of solutions for random regular NAE-SAT

Recent work has made substantial progress in understanding the transitions of random constraint satisfaction problems. In particular, for several of these models, the exact satisfiability threshold has been rigorously determined, confirming predictions of statistical physics. Here we revisit one of these models, random regular k-nae-sat: knowing the satisfiability threshold, it is natural to study, in the satisfiable regime, the number of solutions in a typical instance. We prove here that these solutions have a well-defined free energy (limiting exponential growth rate), with explicit value matching the one-step replica symmetry breaking prediction. The proof develops new techniques for analyzing a certain “survey propagation model” associated to this problem. We believe that these methods may be applicable in a wide class of related problems.

Optimization of the dynamic transition in the continuous coloring problem

Random constraint satisfaction problems (CSPs) can exhibit a phase where the number of constraints per variable α makes the system solvable in theory on the one hand, but also makes the search for a solution hard, meaning that common algorithms such as Monte Carlo (MC) method fail to find a solution. The onset of this hardness is deeply linked to the appearance of a dynamical phase transition where the phase space of the problem breaks into an exponential number of clusters. The exact position of this dynamical phase transition is not universal with respect to the details of the Hamiltonian one chooses to represent a given problem. In this paper, we develop some theoretical tools in order to find a systematic way to build a Hamiltonian that maximizes the dynamic α d threshold. To illustrate our techniques, we will concentrate on the problem of continuous coloring, where one tries to set an angle x i ∈ [0; 2π] on each node of a network in such a way that no adjacent nodes are closer than some threshold angle θ, that is cos(x i − x j )⩽ cos θ. This problem can be both seen as a continuous version of the discrete graph coloring problem or as a one-dimensional version of the Mari–Krzakala–Kurchan model. The relevance of this model stems from the fact that continuous CSPs on sparse random graphs remain largely unexplored in statistical physics. We show that for sufficiently small angle θ this model presents a random first order transition and compute the dynamical, condensation and Kesten–Stigum transitions; we also compare the analytical predictions with MC simulations for values of θ = 2π/q, . Choosing such values of q allows us to easily compare our results with the renowned problem of discrete coloring.

The discrepancy of random rectangular matrices

AbstractA recent approach to the Beck–Fiala conjecture, a fundamental problem in combinatorics, has been to understand when random integer matrices have constant discrepancy. We give a complete answer to this question for two natural models: matrices with Bernoulli or Poisson entries. For Poisson matrices, we further characterize the discrepancy for any rectangular aspect ratio. These results give sharp answers to questions of Hoberg and Rothvoß (SODA 2019) and Franks and Saks (Random Structures Algorithms2020). Our main tool is a conditional second moment method combined with Stein's method of exchangeable pairs. While previous approaches are limited to dense matrices, our techniques allow us to work with matrices of all densities. This may be of independent interest for other sparse random constraint satisfaction problems.

Belief propagation guided decimation algorithms for random constraint satisfaction problems with growing domains

We propose three kinds of belief propagation (BP) guided decimation algorithms using asynchronous updating strategy to solve a prototype of random constraint satisfaction problem with growing domains referred to as model RB. For model RB, the exact satisfiability phase transitions have been established rigorously, and almost all instances are intrinsic hard in the transition region. Finding solutions of a random instance of model RB is very challenging, and the problem size is limited to 102. The BP guided decimation algorithms we proposed are called asynchronous updating belief propagation (ABP) algorithm, asynchronous updating belief propagation* (ABP*) algorithm, and asynchronous updating belief propagation with variable order (VABP) algorithm, respectively. In the BP part of the algorithms, we adopt asynchronous updating strategy to obtain the latest passing messages between constraints and variables, which can improve the convergence of BP equations. We also use a damping factor that adds the old messages with a certain weight into the new messages sent from variables to constraints, to reduce the occurrence of oscillation during the convergence of BP equations. In the ABP algorithm, we compute the marginal probability distribution of all variables according to the messages obtained after the BP equations converge, then select the most biased variable and fix its value on the component with the maximum probability. While the ABP* algorithm considers how to continue the decimation process if the BP equations do not converge. Different from the previous two algorithms, in the VABP algorithm, we first choose a random order of the variables, and then assign values to the variables according to the given order after BP converges. Experimental results suggest that the three kinds of BP guided decimation algorithms appear to be very effective in solving random instances of model RB even when the constraint tightness is close to the theoretical satisfiability threshold. To evaluate the performance of the ABP algorithm, we also provide synchronous updating BP algorithms as a comparison. The entropy of the selected variable at each time step and the average freedom of the variables at different constraint tightness are also discussed. Besides, we analyze the convergence of BP equations and the influence of the order of the selected variables in the decimation process of the BP guided decimation algorithms.

Application of an improved genetic algorithm to Hamiltonian circuit problem

In the last few years, there has been an increasing interest in Random Constraint Satisfaction Problems (CSP) from both experimental and theoretical points of view. To consider a variant instance of the problems, we used a random benchmark. In the present paper, some work has been done to find the shortest Hamiltonian circuit among specified nodes in each superimposed graph (SGs). The Hamiltonian circuit is a circuit that visits each node in the graph exactly once. The Hamiltonian path may be constructed and adjusted according to specific constraints such as time limits. A new constraint satisfaction optimization problem model for the circuit Hamiltonian circuit problem in a superimposed graph has been presented. To solve this issue, we propose amelioration for the genetic algorithm using Dijkstra’s algorithm, where we create the improved genetic algorithm (IGA). To evaluate this approach, we compare the CPU and fitness values of the IGA to the results provided by an adapted genetic algorithm to find the shortest Hamiltonian circuit in a superimposed graph.

The Asymptotics of the Clustering Transition for Random Constraint Satisfaction Problems

Random constraint satisfaction problems exhibit several phase transitions when their density of constraints is varied. One of these threshold phenomena, known as the clustering or dynamic transition, corresponds to a transition for an information theoretic problem called tree reconstruction. In this article we study this threshold for two CSPs, namely the bicoloring of k-uniform hypergraphs with a density $$\alpha $$ of constraints, and the q-coloring of random graphs with average degree c. We show that in the large k, q limit the clustering transition occurs for $$\alpha = \frac{2^{k-1}}{k} (\ln k + \ln \ln k + \gamma _{\mathrm{d}} + o(1))$$ , $$c= q (\ln q + \ln \ln q + \gamma _{\mathrm{d}}+ o(1))$$ , where $$\gamma _{\mathrm{d}}$$ is the same constant for both models. We characterize $$\gamma _{\mathrm{d}}$$ via a functional equation, solve the latter numerically to estimate $$\gamma _{\mathrm{d}} \approx 0.871$$ , and obtain an analytic lowerbound $$\gamma _{\mathrm{d}} \ge 1 + \ln (2 (\sqrt{2}-1)) \approx 0.812$$ . Our analysis unveils a subtle interplay of the clustering transition with the rigidity (naive reconstruction) threshold that occurs on the same asymptotic scale at $$\gamma _{\mathrm{r}}=1$$ .

Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion

We investigate the clustering transition undergone by an exemplary random constraint satisfaction problem, the bicoloring of k-uniform random hypergraphs, when its solutions are weighted non-uniformly, with a soft interaction between variables belonging to distinct hyperedges. We show that the threshold α d(k) for the transition can be further increased with respect to a restricted interaction within the hyperedges, and perform an asymptotic expansion of α d(k) in the large k limit. We find that , where the constant γ d is strictly larger than for the uniform measure over solutions.

The replica symmetric phase of random constraint satisfaction problems

AbstarctRandom constraint satisfaction problems play an important role in computer science and combinatorics. For example, they provide challenging benchmark examples for algorithms, and they have been harnessed in probabilistic constructions of combinatorial structures with peculiar features. In an important contribution (Krzakala et al. 2007, Proc. Nat. Acad. Sci.), physicists made several predictions on the precise location and nature of phase transitions in random constraint satisfaction problems. Specifically, they predicted that their satisfiability thresholds are quite generally preceded by several other thresholds that have a substantial impact both combinatorially and computationally. These include the condensation phase transition, where long-range correlations between variables emerge, and the reconstruction threshold. In this paper we prove these physics predictions for a broad class of random constraint satisfaction problems. Additionally, we obtain contiguity results that have implications for Bayesian inference tasks, a subject that has received a great deal of interest recently (e.g. Banks et al. 2016, Proc. 29th COLT).

Biased landscapes for random constraint satisfaction problems

The typical complexity of constraint satisfaction problems (CSPs) can be investigated by means of random ensembles of instances. The latter exhibit many threshold phenomena besides their satisfiability phase transition, in particular a clustering or dynamic phase transition (related to the tree reconstruction problem) at which their typical solutions shatter into disconnected components. In this paper we study the evolution of this phenomenon under a bias that breaks the uniformity among solutions of one CSP instance, concentrating on the bicoloring of k-uniform random hypergraphs. We show that for small k the clustering transition can be delayed in this way to higher density of constraints, and that this strategy has a positive impact on the performances of simulated annealing algorithms. We characterize the modest gain that can be expected in the large k limit from the simple implementation of the biasing idea studied here. This paper contains also a contribution of a more methodological nature, made of a review and extension of the methods to determine numerically the discontinuous dynamic transition threshold.

Displacement Convexity in Spatially Coupled Scalar Recursions

We introduce a technique for the analysis of general spatially coupled systems that are governed by scalar recursions. Such systems can be expressed in variational form in terms of a potential function. We show, under mild conditions, that the potential function is displacement convex and that the minimizers are given by the fixed points (FPs) of the recursions. Furthermore, we give the conditions on the system such that the minimizing FP is unique up to translation along the spatial direction. The condition matches with that of Kudekar et al. [20] for the existence of spatial FPs. Displacement convexity applies to a wide range of spatially coupled recursions appearing in coding theory, compressive sensing, random constraint satisfaction problems, as well as statistical-mechanics models. We illustrate it with applications to low-density parity-check (LDPC) and generalized LDPC codes used for the transmission on the binary erasure channel or general binary memoryless symmetric channels within the Gaussian reciprocal channel approximation as well as compressive sensing.

Decoding from Pooled Data: Sharp Information-Theoretic Bounds

Consider a population consisting of n individuals, each of whom has one of d types (e.g. their blood type, in which case d=4). We are allowed to query this database by specifying a subset of the population, and in response we observe a noiseless histogram (a d-dimensional vector of counts) of types of the pooled individuals. This measurement model arises in practical situations such as pooling of genetic data and may also be motivated by privacy considerations. We are interested in the number of queries one needs to unambiguously determine the type of each individual. In this paper, we study this information-theoretic question under the random, dense setting where in each query, a random subset of individuals of size proportional to n is chosen. This makes the problem a particular example of a random constraint satisfaction problem (CSP) with a "planted" solution. We establish almost matching upper and lower bounds on the minimum number of queries m such that there is no solution other than the planted one with probability tending to 1 as n tends to infinity. Our proof relies on the computation of the exact "annealed free energy" of this model in the thermodynamic limit, which corresponds to the exponential rate of decay of the expected number of solution to this planted CSP. As a by-product of the analysis, we show an identity of independent interest relating the Gaussian integral over the space of Eulerian flows of a graph to its spanning tree polynomial.

Phase transitions of random constraint satisfaction problems

Phase transitions of random constraint satisfaction problems